dqbd/tiktoken

Preview

Will there be support for deno deploy + esm.sh
#22 · deepak-coding-art opened on Mar 17, 2023
8
@sveltejs/adapter-cloudflare: cloudflare worker (pages) with vite: "default" is not exported by module
#37 · happysalada opened on May 8, 2023
12

Labels Milestones New issue

Expose getEncodingNameForModel in wasm version?

#123

· jens-f opened on Nov 14, 2024

· Updated on Nov 14, 2024

recursive use of an object detected which would lead to unsafe aliasing in rust

#119

· mogery opened on Aug 22, 2024

· Updated on Nov 1, 2024

2

React native error SyntaxError: Invalid RegExp: Invalid escape] in react native expo.

#110

· djaffer opened on Jul 16, 2024

· Updated on Oct 14, 2024

1

Cloudflare worker Uncaught ReferenceError: FinalizationRegistry is not defined

#46

· EiffelFly opened on May 24, 2023

· Updated on Oct 2, 2024

8

Will there be support for deno deploy + esm.sh

#22

· deepak-coding-art opened on Mar 17, 2023

· Updated on Oct 2, 2024

8

Error: null pointer passed to rust

#69

· emeeks opened on Sep 6, 2023

· Updated on Sep 30, 2024

4

NPM package is very huge

#68

· anzemur opened on Aug 29, 2023

· Updated on Sep 28, 2024

6

missing example code for js-tiktoken

#102

· coding22 opened on May 9, 2024

· Updated on Sep 20, 2024

1

Add support for the newest Open AI embeddings models

#90

· AleksTrpkovski opened on Feb 9, 2024

· Updated on Jul 2, 2024

2

Performance

#99

· steebchen opened on Apr 15, 2024

· Updated on Apr 15, 2024

Cannot find module error

#75

· thdoan opened on Oct 17, 2023

· Updated on Mar 15, 2024

1

Add decode_with_offsets function (or PR guidance request)

#94

· Sam1320 opened on Mar 11, 2024

· Updated on Mar 11, 2024

Memory access out of bounds

#84

· maxdeichmann opened on Jan 17, 2024

· Updated on Mar 1, 2024

3

Cloudflare worker Uncaught ReferenceError: FinalizationRegistry is not defined

#92

· TranquilMarmot opened on Feb 13, 2024

· Updated on Feb 18, 2024

1

'TextEncoder' is not defined - MongoDB Functions

#91

· georgeherby opened on Feb 11, 2024

· Updated on Feb 11, 2024

1

Crash on React Native: `SyntaxError: Invalid RegExp: Invalid escape`

#82

· julian-hecker opened on Dec 1, 2023

· Updated on Jan 11, 2024

2

How to count tokens for functions defined in the prompt?

#78

· vrde opened on Oct 21, 2023

· Updated on Oct 26, 2023

1

Support fine tunned models

#76

· italojs opened on Oct 17, 2023

· Updated on Oct 18, 2023

1

What does free actually do?

#72

· disbelief opened on Sep 19, 2023

· Updated on Oct 10, 2023

9

Raise error when concurrent to call new Tiktoken

#70

· qkxie opened on Sep 12, 2023

· Updated on Sep 12, 2023

2

When running 1.0.7 there is a Missing tiktoken_bg.wasm error on AWS Lambda typescript

#61

· haibh-wanosoft opened on Jul 10, 2023

· Updated on Aug 7, 2023

3

Fatal crash: BasicBlock requested for unrecognized address

#62

· SamStenner opened on Jul 13, 2023

· Updated on Aug 7, 2023

1

When I look at cl100k_base.json, I don't see the special tokens like <|im_start|>, but when I add them manually I get an error from the wasm

#65

· Evertt opened on Aug 5, 2023

· Updated on Aug 7, 2023

2

[js-tiktoken] memory leak

#50

· louis030195 opened on Jun 13, 2023

· Updated on Jun 16, 2023

3

Automatically determine the environment

#43

· louis030195 opened on May 16, 2023

· Updated on May 18, 2023

3

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Expose getEncodingNameForModel in wasm version?

recursive use of an object detected which would lead to unsafe aliasing in rust

React native error SyntaxError: Invalid RegExp: Invalid escape] in react native expo.

Cloudflare worker Uncaught ReferenceError: FinalizationRegistry is not defined

Will there be support for deno deploy + esm.sh

Error: null pointer passed to rust

NPM package is very huge

missing example code for js-tiktoken

Add support for the newest Open AI embeddings models

Performance

Cannot find module error

Add decode_with_offsets function (or PR guidance request)

Memory access out of bounds

Cloudflare worker Uncaught ReferenceError: FinalizationRegistry is not defined

'TextEncoder' is not defined - MongoDB Functions

Crash on React Native: `SyntaxError: Invalid RegExp: Invalid escape`

How to count tokens for functions defined in the prompt?

Support fine tunned models

What does free actually do?

Raise error when concurrent to call new Tiktoken

When running 1.0.7 there is a Missing tiktoken_bg.wasm error on AWS Lambda typescript

Fatal crash: BasicBlock requested for unrecognized address

When I look at cl100k_base.json, I don't see the special tokens like <|im_start|>, but when I add them manually I get an error from the wasm

[js-tiktoken] memory leak

Automatically determine the environment

Issues

Search results